We present the development of a VoIP quality of service (QoS) measurement system that enables operators to diagnose a QoS degradation segment. Our system uses a flow-based passive measurement method to fulfill the requirement for QoS measurement in large-scale IP networks. In particular, we adopt an access control list (ACL)-based filtering function that selects traffic to monitor and develop a function for correlating signals and media data records. This correlation function is required to dynamically configure ACL-based filtering for monitoring media streams whose port numbers are determined by a signaling protocol. To improve the scalability of existing measurement systems, we also develop a hardware-based filtering engine on a commercial switch as well as a mediation box that performs QoS calculation based on traffic records exported by the engine in a distributed manner. We demonstrate the feasibility of the measurement system by evaluating a prototype system.

We propose a method of IP traffic management where the TCP performance at a bottleneck link is estimated from monitored data about the behavior of the number of active flows versus link utilization, which are both easy to measure. This method is based on our findings that (i) TCP performance remains constant as long as the link utilization is below some threshold value, but becomes degraded when it exceeds this value and (ii) the number of active flows increases linearly with link utilization up to the same value, and the increase becomes nonlinear above it. Though this threshold may vary depending on traffic/network conditions, our method requires neither predetermination of a threshold on the basis of assumed traffic conditions nor direct measurement of TCP performance.

We introduce the notion of hierarchical aggregate entropy and apply it to identify DNS client hosts that wastefully consume server resources. Entropy of DNS query traffic can capture client query patterns, e.g., the concentration of queries to a specific domain or dispersion to a large domain name space. However, entropy alone cannot capture the spatial structure of the traffic. That is, even if queries disperse to various domains but concentrate in the same upper domain, entropy among domain names provides no information on the upper domain structure, which is an important characteristic of DNS traffic. On the other hand, entropies of aggregated upper domains do not have detailed information on individual domains. To overcome this difficulty, we introduce the notion of hierarchical aggregate entropy, where queries are recursively aggregated into upper domains along the DNS domain tree, and their entropies are calculated. Thus, this method enables us to analyze the spatial characteristics of DNS traffic in a multi-resolution manner. We calculate the hierarchical aggregate entropies for actual DNS heavy-hitters and observed that the entropies of normal heavy-hitters were concentrated in a specific range. On the basis of this observation, we adopt the support vector machine method to identify the range and to classify DNS heavy-hitters as anomalous or normal. It is shown that with hierarchical aggregate entropy can halve the classification error compared to non-hierarchical entropies.

To efficiently use network resources, internet service providers need to conduct traffic engineering that dynamically controls traffic routes to accommodate traffic change with limited network resources. The performance of traffic engineering (TE) depends on the accuracy of traffic prediction. However, the size of traffic change has been drastically increasing in recent years due to the growth in various types of network services, which has made traffic prediction difficult. Our approach to tackle this issue is to separate traffic into predictable and unpredictable parts and to apply different control policies. However, there are two challenges to achieving this: dynamically separating traffic according to predictability and dynamically controlling routes for each separated traffic part. In this paper, we propose a macroflow-based TE scheme that uses different routing policies in accordance with traffic predictability. We also propose a traffic-separation algorithm based on real-time traffic analysis and a framework for controlling separated traffic with software-defined networking technology, particularly OpenFlow. An evaluation of actual traffic measured in an Internet2 network shows that compared with current TE schemes the proposed scheme can reduce the maximum link load by 34% (at the most congested time) and the average link load by an average of 11%.

Network equipment, such as routers, switches, and RADIUS servers, generate various log messages induced by network events such as hardware failures and protocol flaps. In large production networks, analyzing the log messages is crucial for diagnosing network anomalies; however, it has become challenging due to the following two reasons. First, the log messages are composed of unstructured text messages generated in accordance with vendor-specific rules. Second, network events that induce the log messages span several geographical locations, network layers, protocols, and services. We developed a method to tackle these obstacles consisting of two techniques: statistical template extraction (STE) and log tensor factorization (LTF). The former leverages a statistical clustering technique to automatically extract primary templates from unstructured log messages. The latter builds a statistical model that collects spatial-temporal patterns of log messages. Such spatial-temporal patterns provide useful insights into understanding the impact and patterns of hidden network events. We evaluate our techniques using a massive amount of network log messages collected from a large operating network and confirm that our model fits the data well. We also investigate several case studies that validate the usefulness of our method.